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(12) to enable a control function associated with a location of the display screen (16) in response to the corresponding location of 
the touch pad (12) being touched for controlling the HE device. The display screen may include on-screen keyboards, control panels 
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REMOTE CONTROL HAVING TOUCH 
PAD TO SCREEN MAPPING 

Technical Field 

The present invention generally relates to remote controls for 
5 controlling home entertainment devices and, more particularly, to a remote control 
having a touch pad mapped to the screen of a home entertainment (HE) device to 
create a virtual touch screen for enabling an operator to control the HE device. 

Background Art 

Other than being remote, typical home entertainment (HE) device 
10 remote controls are generally despised by human operators for a variety of 
reasons. Typical HE remote controls are based on a button-centric paradigm and 
usually contain too many buttons. This crowded geography causes considerable 
confusion and intimidation and makes it difficult to find the desired button. It is 
often dark in the room in which HE remote controls are used. This makes it 
15 difficult or impossible to read the legends on the crowded button layout, 

Nomial HE viewing takes place at a distance of two meters or more 
and the display being viewed is usually quite large such as a television (TV). The 
legends on HE remote controls are usually twelve point type or smaller. For 
many operators, changing viewing distance requires changing glasses or putting 
20 on reading glasses. 

Some remote controls have a trackball or button pointing device to 
control a cursor on a TV screen. This approach has drawbacks in that the low 
resolution of normal TVs requires a cursor that is large. The cursor also moves 
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in a jerky style because of the low refresh rate of the TV screen. Other remote 
controls have tried to solve the problems associated with typical remote controls 
by using a display in the remote control. The display is changed as required. 
However, the local display brings with it a new set of problems. An operator has 
5 to read the display then press the appropriate area to select an item or another 
menu. The display also is costly and requires programming for each system or bi- 
directional communications. 

Enhanced or interactive TV and related applications require the 
extensive use of graphic user interfaces (GUIs) and on-screen displays/menus. An 

10 enhanced TV is a TV configured for cable video progranuning , Internet browsing, 
Internet telephony, electronic mail, video cassette recording, audio programming, 
and the like. The operator has to navigate through various menus to use the 
enhanced TV. Using up, down, right, and left arrow keys to navigate these menus 
is difficult, slow, and frustrating. The increasing number of television chaimels 

15 has given rise to the electronic programming guide (EPG). Because an EPG is a 
dense grid of selections, using arrow keys to navigate is even more difficult. 

Interactive TV also requires text and graphics entry. The current 
solution for text entry, a wireless keyboard, is undesirable in a living room 
because many people cannot type, the keyboard does not fit the decor of the living 
20 room, there is no place to set the keyboard down to type on it, and people simply 
refuse to have computer related material in living rooms. In addition, typing is 
associated with work and is not desired to be in a room devoted to relaxation. 

What is needed is a remote control having a touch pad mapped to 
an HE device display screen to create a virtual touch screen for enabling an 
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Operator to control the HE device. In effect, the touch pad would provide an 
operator with the ability to virtually touch the HE device display screen to control 
the HE device without having to actually physically touch the HE device. 
Touching the HE device display screen to control the HE device is the most 
5 intuitive interface. Further, the advantages of using a touch pad to control the HE 
device include allowing the operator to not need to look at the remote control 
during operation. 

Summary of the Invention 

Accordingly, it is an object of the present invention to provide a 
10 remote control having a touch pad mapped to a home entertainment (HE) device 
display screen to create a virtual touch screen for enabling an operator to control 
the HE device without looking at the remote control. 

It is another object of the present invention to provide a remote 
control having a touch pad mapped to a control panel displayed on the HE device 
15 display screen such that each location of the touch pad corresponds to a respective 
portion of the control panel. 

It is a further object of the present invention to provide a remote 
control having a touch pad mapped to an on-screen keyboard displayed on the HE 
device display screen such that each location of the touch pad corresponds to a 
20 respective entry of the on-screen keyboard. 

It is still another object of the present invention to provide a remote 
control having a touch pad mapped to an electronic program guide (EPG) 
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displayed on the HE device display screen such that each location of the touch pad 
corresponds to a grid area of the EPG. 

It is still a further object of the present invention to provide a 
remote control having a touch pad mapped to an Internet page displayed on the HE 
5 device display screen such that each location of the touch pad corresponds to a link 
of the Internet page. 

It is still yet another object of the present invention to provide a 
remote control having a touch pad and control buttons used in conjunction to 
enable control functions for controlling an HE device. 

10 It is still yet a further object of the present invention to provide a 

remote control having a touch pad mapped to a graphical entry screen such that 
each location of the touch pad corresponds to a respective location of the graphical 
entry screen. 

In carrying out the above objects and other objects, an embodiment 
15 of the present invention provides a remote control and an associated method for 
controlling a home entertainment (HE) device having a display screen. The 
remote control includes a touch pad mapped to the display screen of the HE device 
as a function of the ratio of the areas of the touch pad and the display screen such 
that each location of the touch pad corresponds to a respective location of the 
20 display screen. The remote controller further includes a controller operable with 
the touch pad to enable a control function associated with a location of the display 
screen in response to the corresponding location of the touch pad being touched 
for controlling the HE device. The display screen may include on-screen 
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keyboards, control panels and menus, an Internet page, an electronic program 
guide, and the lii^e. 

Further, in carrying out the above objects and other objects, the 
present invention provides a remote control for controlling a television. The 
5 remote control includes a touch pad and a plurality of control buttons. The touch 
pad and the control buttons are used in conjunction to enable control functions for 
controlling the television. 

These and other features, aspects, and embodiments of the present 
invention will become better understood with regard to the following description, 
10 appended claims, and accompanying drawings. 

Brief Description of the Drawings 

FIG. 1 illustrates a block diagram of a remote control for 
controlling a home entertainment (HE) device in accordance with the present 
invention; 

15 FIG. 2 illustrates the absolute position mapping of the touch pad 

with respect to the display screen of the HE device in accordance with the present 
invention; 

FIG. 3 illustrates the absolute control mapping of the touch pad 
with respect to the display screen in accordance with the present invention; 
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FIG. 4 illustrates a perspective view of a remote control for 
controlling HE devices in accordance with the present invention; 

FIG. 5A illustrates a main menu displayed on the display screen of 
the HE device; 

5 FIG. 5B illustrates the virtual control areas of the touch pad 

corresponding to the controls of the main menu displayed on the display screen of 
the HE device; 

FIG. 6 illustrates the main menu having a highlighted VCR mode 

button; 

10 FIG. 7A illustrates an electronic program guide (EPG) displayed 

on the display screen of the HE device when the HE device is in the cable 
programming mode; 

FIG. 7B illustrates the virtual control areas of the touch pad 
corresponding to the grid areas of the EPG displayed on the display screen of the 
15 HE device; 

FIG. 8A illustrates a cable program and cable control panel 
displayed on ttie display screen of the HE device when the HE device is in the 
cable programming mode; 
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FIG. 8B illustrates the virtual control areas of the touch pad mapped 
absolutely to the cable control panel displayed on the display screen of the HE 

device; 

FIG. 9 illustrates a VCR program and VCR control panel displayed 
5 on the display screen of the HE device when the HE device is in the VCR mode; 

FIGi 10 illustrates an Internet page and on-screen keyboard 
displayed on the display screen of the HE device when the HE device is in the 
Internet browsing mode; 

FIG. 11 illustrates an Internet page and a graphical entry screen 
10 displayed on the display screen of the HE device when the HE device is in the 
Internet browsing mode; 

FIG. 12A illustrates an audio receiver control panel displayed on 
the display screen of the HE device when the HE device is in the audio receiver 
mode; 

15 FIG. 12B illustrates the virtual control areas of the touch pad 

mapped to the audio receiver control panel displayed on the HE device; 

FIG. 13 illustrates an Internet telephony control panel displayed on 
the display screen of the HE device when the HE device is in the Internet 
telephony mode; 
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FIG. 14 illustrates an on-screen numerical keyboard displayed on 
the display screen of the HE device when the HE device is in ttie Internet 
telephony mode; 

PIG, 15 illustrates an email panel displayed on the display screen 
5 of the HE device when the HE device is in the email mode; 

FIG. 16 illustrates an email messaging panel and on-screen panel 
displayed in the display screen of HE device; 

FIG. 17 illustrates the email messaging panel having a writing area 
displayed in the display screen of the HE device; 

10 FIG. 18 illustrates a flow chart describing operation of the 

switching between absolute and relative modes of the display screen of the HE 
device; and 

FIG. 19 illustrates a block diagram a remote control system in 
accordance with the present invention. 

15 Best Modes For Carrying Out The Invention 

Referring now to FIG. 1, a block diagram of a remote control 
system 10 for controlling a home entertainment (HE) device in accordance with 
the present invention is shown. HE remote control system 10 includes a touch pad 
12 on a remote control 13, a controller 14, and a display screen 16. Touch pad 
20 12 includes a touch pad surface area for an operator to touch. Touch pad 12 
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generates a signal in response to touching by an operator on the touch pad. The 
signal is indicative of the location of the touch on touch pad 12. The signal may 
also be indicative of the duration and the pressure of the touch on touch pad 12 for 
each location being touched. 

5 Touch pad 12 is operable with display screen 16 such that the area 

of the touch pad is absolutely mapped to the area of the display screen. This 
means that each portion of touch pad 12 corresponds to a respective portion of 
display screen 16. For instance, the upper left portion of touch pad 12 
corresponds to the upper left portion of display screen 16. Similarly, the middle 
10 bottom portion of touch pad 12 corresponds to the middle bottom portion of 

display screen 16. Touch pad 12 is in an absolute mode when the area of the 
touch pad is absolutely mapped to the area of display screen 16. 

In operation, the area of touch pad 12 is mapped to various control 
panels and menus displayed on display screen 16. This allows the operator to 
15 manipulate touch pad 12 to select control function entries of the panels and menus 
displayed on display screen 16 while remaining visually focused on the display 
screen. 

Preferably, display screen 16 has a much larger area than the area 
• of touch pad 12. The absolute mapping between touch pad 12 and display screen 
20 16 is scaled as a function of the ratio of the areas. Display screen 16 is preferably 
some or all of the display screen area of a HE device 17 such as a television (TV) 
screen, monitor, computer screen, or the like. HE device is part of an HE system 
that may include other HE devices such as a sound system, a player/recorder and 
the like. Display screen 16 may include a movable object such as a cursor 18 for 
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touch pad 12 to operate in a relative mode. Display screen 16 may be separated 
from HE device and coupled directly to touch pad 12. 

Controller 14 is operable with touch pad 12 for receiving a signal 
19 from the touch pad in response to an operator touching the touch pad. In the 

5 absolute mode, controller 14 highlights a portion of display screen 16 in response 

to a corresponding portion of touch pad 12 being touched. Controller 14 
highlights a portion of display screen 16 to enable a control function 
corresponding to the highlighted portion of the display screen for controlling the 
HE device. Instead of highlighting portions of display screen 16, controller 14 

10 may user color change, outlining, or other ways of distinguishing the selected 

portions of display screen 16. To enable touch pad to display screen naapping, 
remote control system 10 generally includes the following hardware: touch pad 
12, conununications, receiver, driver level software, video source selector, video 
overlay generator, and video to TV converter as is known in the art. 

15 In the relative mode, controller 14 moves cursor 18 on display 

screen 16 to the location on display screen corresponding to the location of the 
touch on touch pad 12 in response to an operator touching the touch pad. 
Controller 14 controls the HE device to enable a control fimction corresponding 
to the location of cursor 18 on display screen 16 in response to an operator 

20 touching touch pad 12. Controller 14 may be coupled directly or remotely located 
from touch pad 12. Typically, touch pad 12 transuMts infrared (DR) or radio 
frequency (RF) signals to communicate with controller 14. Traditional pointing 
devices such as a computer mouse use relative pointing letting the operator move 
a cursor from one place to another place on a display screen. 
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In the absolute mode, controller 14 highlights the portion of display 
screen 16 corresponding to the respective portion of touch pad 12 being touched 
independent of the display screen portion highlighted prior to the touch pad being 
touched. Wherever the operator touches touch pad 12 the corresponding portion 
5 of display screen 16 is highlighted regardless of where the operator touched the 

touch pad previously. That is, the touching movement on touch pad 12 is mapped 
absolutely on to display screen 16. 

Referring now to FIG. 2, the absolute position mapping of touch 
pad 12 with respect to display screen 16 will now be described in further detail. 

10 As mentioned above, touch pad 12 is operable with display screen 16 such that the 

area of the touch pad is absolutely mapped to the area of either a portion of the 
display screen or the entire display screen. As an example, it will be assumed that 
the area of touch pad 12 is absolutely mapped to the entire area of display screen 
16. Thus, each point of touch pad 12 has a corresponding absolute point on 

15 display screen 16. Touch pad 12 has an area defined by a width Wp and a height 

hp. Display screen 16 has an area defined by a width w^ and a height h^. A 
selected point of touch pad 12 such as point Xp, yp has a corresponding absolute 
point Xd, yd- The corresponding absolute point x^, ya of display screen 16 is 
related to the selected point Xp, yp of touch pad 12 in accordance with the 

20 following equations: 

Xd = Xp * (Wd / Wp) 

Yd = yp * (hd / hp)- 

That is, the x coordinate of the corresponding absolute point of 
display screen 16 (Xj) is equal to the x coordinate of the selected point of touch 

-11- 

JDOCID: <WO ^0178054A1_I_> 



wo 01/78054 PCT/USOl/11682 



pad 12 (Xp) multiplied by the ratio between the widths of the display screen (w^) 
and the touch pad (Wp). Similarly, the y coordinate of the corresponding absolute 
point of display screen 16 (y^) is equal to the y coordinate of the selected point of 
touch pad 12 (y^) multiplied by the ratio between the heights of the display screen 
5 (ha) and the touch pad (hp). As a result, each point of touch pad 12 has a 

corresponding absolute point on display screen 16. 

If touch pad 12 is mapped to only a portion of the entire area of 
display screen 16 then the x coordinate of the corresponding absolute point of the 
portion of the display screen (x^) is equal to the x coordinate of the selected point 
10 of touch pad 12 (Xp) multiplied by the ratio between the widths of the portion of 

the display screen and the touch pad. The y coordinate of the corresponding 
absolute point of the portion of display screen 16 (y^) is equal to the y coordinate 
of the selected point of touch pad 12 (y^ multiplied by the ratio between the 
heights of the portion of the display screen and the touch pad. 

15 Referring now to FIG. 3 , the absolute control mapping of touch pad 

12 with respect to display screen 16 will now be described in further detail. As 
mentioned above, the area of touch pad 12 is mapped to various control panels and 
menus displayed on display screen 16. This allows the operator to manipulate 
touch pad 12 to select control function entries of the panels and menus displayed 

20 on display screen 16 while remaining visually focused on the display screen. The 
area of touch pad 12 is broken down into a plurality of virtual control areas (VCl, 
VC2, etc.). Display screen 16 (or a portion of the display screen) displays a 
plurality of virtual controls (VCl, VC2, etc.). Each virtual control area of touch 
pad 12 corresponds to a respective virtual control of display screen 16. For 

25 instance, virtual control area VCl of touch pad 12 corresponds to virtual control 
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VCl of display screen 16 and virtual control area VC2 of the touch pad 
corresponds to virtual control VC2 of the display screen. 

The entire area of touch pad 12 is mapped to virtual controls 
displayed on display screen 16 such that each point of the touch pad corresponds 
5 to a virtual control displayed on the display screen. Each virtual control area of 
touch pad 12 corresponds to a respective virtual control of display screen 16 as a 
function of the positions of the virtual control areas on the touch pad and the 
positions of the virtual controls on the display screen. For instance, virtual 
control area VCl of touch pad 12 is in the upper left comer of the touch pad and 
10 corresponds to virtual control VCl of display screen 16 located in the upper left 

comer of the display screen. Virtual control area VC6 of touch pad is in tiie right 
side of touch pad 12 and corresponds to virtual control VC6 of display screen 16 
located in the right side of the display screen. 

There are three virtual controls in the left side of display screen 16 
15 (VCl, VC2, and VC3) so the left side of touch pad 12 includes three 

corresponding virtual control areas (VCl , VC2, and VC3). There are two virtual 
controls in the middle of display screen 16 (VC4, VC5) so the middle of touch pad 
12 includes two corresponding virtual control areas (VC4, VC5). Finally, there 
is one virtual control in the right side of display screen 16 (VC6) so the right side 
20 of touch pad 12 includes one corresponding virtual control area (VC6). 

In operation, the operator touches a point of touch pad 12 falling 
within a certain virtual control area. In response, the virtual control of display 
screen 16 corresponding to the certain virtual control area is selected or actuated. 
HE device 17 is then controlled as a function of the selected control. 
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Referring now to FIGS. 4 through 17, the operation of remote 
control system 10 in a typical enhanced TV environment will now be described 
according to an embodiment of the present invention. An enhanced TV is a TV 
system configured for cable video programming, Internet browsing, telephony, 
5 email messaging, video cassette recording (VCR), audio programming, and the 

like. 

A perspective view of a remote control 13 for controlling HE 
devices in accordance with the present invention is shown in FIG. 4. Remote 
control 13 includes a touch pad 12 and a plurality of control buttons 24. An 
operator uses touch pad 12 and control buttons 24 in conjunction to operate with 
controller 14 for selecting a control function for controlling a HE device. In 
general, an operator uses touch pad 12 to highlight a selected portion of display 
screen 16. The operator then uses control buttons 24 to select the highlighted 
portion of display screen 16. Control buttons 24 include a select button 24a, an 
on-screen control button 24b, a main screen button 24c, an electronic program 
guide button 24d, a write button 24e, a type button 24f, a talk button 24g, a 
volume mute button 24h, and a power button 24i. 

FIG. 5A illustrates a main menu 30 having a plurality of virtual 
controls displayed on display screen 16 of the HE device. Main menu 30 includes 
20 a visual panel 32 for visually showing a video image on the HE device. For 

instance, visual panel 32 shows cable or video programming. Main menu 30 
further includes an HE device operating panel 34. HE device operating panel 34 
includes a plurality of mode buttons 36. Main menu 30 further includes an 
electronic mail (email) panel 38 and an Internet telephony panel 40. 
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Each of mode buttons 36 corresponds to a mode of the HE device. 
For instance, modes of the HE device include an Internet browser mode, a cable 
mode, a VCR mode, and an audio receiver mode. In the Internet browser mode, 
the HE device functions as an access device for Internet communications. For 
5 Internet communications the entire area of display screen 16 displays Internet 

pages. In the cable mode, the HE device functions as a television for receiving 
video/audio cable programming. For cable progranuning the entire area of display 
screen 16 displays video cable progranuning. In the VCR mode, the HE device 
functions as a VCR and the entire area of display screen 16 shows video 
10 programming. In the audio receiver mode, the HE device functions as a stereo 

receiver for playing audio signals. In FIG. 5 A, the HE device is in the cable 
mode as evidenced by cable mode button 36 being bigger than the other mode 
buttons 36. Visual panel 32 displays cable programming of a selected cable 
channel for the operator to view. 

15 Referring now to FIG. 5B with continual reference to FIG. 5A, the 

area of main menu 30 is mapped to the area of touch pad 12. Each portion of 
main menu 30 corresponds to a respective portion of touch pad 12. The operator 
touches touch pad 12 using a finger or a stylus to highlight panels 32, 34, 38, and 
40 of display screen 16. For instance, the operator touches virtual control area 

20 32a in the upper left portion of touch pad 12 to highlight visual panel 32, virtual 
control areas 36a hi the bottom portion of the touch pad to highlight operating 
panel 34, vurtual control area 40a in the mid right portion of the touch pad to 
highlight Internet telephony panel 40, and virtual control area 38a in the upper 
right portion of the touch pad to highlight email panel 38. 
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When the operator touches a portion of touch pad 12 the panel 
closest to the corresponding touched portion of the touch pad is highlighted. If the 
operator moves his finger on touch pad 12 from a first panel towards a second 
panel the second panel is highlighted as soon as the operator's finger is closer to 
5 the second panel than to the first panel. To activate a panel the operator may 
either press select button 24a of remote control 13, press harder on touch pad 12, 
depress hard enough to activate a switch under the touch pad, quickly tap on touch 
pad, or the like. 

FIG. 6 illustrates main menu 30 having a highlighted VCR mode 
button 36. In addition to highlighting panels 32, 34, 38, and 40 of main menu 16, 
an operator may highlight a mode button 36 of operating panel 34 by touching the 
portion of touch pad 12 corresponding to the location of the mode button on 
display screen 16. For example, if the operator, touches the bottom middle ri^t 
portion of touch pad 12, then VCR mode button 36 is highlighted as shown in 
FIG. 6. The operator may then select the highlighted VCR mode button 36 by 
pressing select button 24a on remote control 13 . The entire display screen 16 then 
displays VCR progranuning in place of main menu 30 (not shown). 

Upon selecting cable mode button 36, display screen 16 replaces 
main menu 30 with video cable programming. For instance, the entire area of 
20 display screen 16 displays a video picture and functions like a typical TV screen. 

While viewing the video cable programming the operator may desire to know 
which other programs are being provided on oflier cable channels. An electronic 
program guide (EPG) is a channel guide that enables a user to view a listing of the 
programs provided on the cable channels. The operator may access an EPG by 

-16- 



SDOCID: <WO_0178054A1J_> 



10 



15 



wo 01/78054 



PCT/USOl/11682 



pressing guide button 24d on remote control 13 or a virtual control on display 
screen 16. 

In response to guide button 24d being pressed display screen 16 
displays an EPG 50 over cable video programming screen 52 as shown in FIG. 
5 7A. Cable programming screen 52 displays the cable program. EPG 50 includes 
a plurality of grids such as channel grids 54 for listing cable channels and program 
title grids 56 for listing programs associated with the cable channels during given 
times. Highlighted program title grid 62 identifies the program playing on cable 
video prograrmning screen 52. EPG 50 further includes day grids 58 for the 
10 operator to select a day to view the available programs for the selected day. EPG 
50 further includes arrow keys 60 to move sets of grids of the EPG up and down 
or sideways. 

Referring now to FIG. 7B with continual reference to FIG. 7A, the 
virtual control areas of touch pad 12 is mapped absolutely to EPG 50 and each 

15 grid of EPG 50 corresponds to a respective portion of touch pad 12. The operator 

touches a portion of touch pad 12 to highlight the corresponding grid of EPG 50. 
For example, the operator touches a virtual control area 54a in the left hand side 
of touch pad 12 to highlight a corresponding channel grid 54 of EPG 50. The 
operator may touch a virtual control area 56a of touch pad 12 to highlight a 

20 corresponding program title grid 56 of EPG 50. Similarly, virtual control areas 
58a, 60a, 62a, and 64a are mapped to corresponding grids of EPG 50. As the 
operator touches different portions of touch pad 12 the corresponding grids of 
EPG 50 are highlighted. 



-17- 



SCXX^ID: <WO_0178054A1J_> 



wo 01/78054 



PCT/USOl/11682 



In operation, the operator may touch virtual control area 64a in the 
middle portion of touch pad 12 to highlight program title grid 64. The operator 
may then select the program of program title grid 64 to view the program by 
pressing select button 24a of remote control 13. In response to program title grid 
5 64 being selected the cable channel is changed to the associated channel and the 

program of program title grid 64 is displayed on the entire display screen 16. 
Concurrently, EPG 50 disappears from display screen 16. Instead of selecting the 
program of program title grid 64 to view the program the operator may select 
program title grid 64 to view information associated with the program by pressing 
10 guide button 24d of remote control 13 . The information may include a description 

of the program, running time, and the like. 

FIG. 8 A illustrates a cable program being displayed on cable video 
programming screen 52 of display screen 16. While viewing titie cable program 
on cable video programming screen 52 the operator may access an on-screen 
15 control panel 70 by selecting on-screen control button 24b of remote control 13. 

On-screen control panel 70 includes a numerical key pad 72, cable chaimel control 
arrows 74, and volume control arrows 76. 

Referring now to FIG. 8B with continual reference to FIG. 8A, the 
virtual control areas of touch pad 12 is mapped absolutely to on-screen control 

20 panel 70 and each portion of the on-screen control panel corresponds to a 
respective portion of the touch pad. The operator touches a portion of touch pad 
12 to highlight the corresponding portion of on-screen control panel 70. For 
example, the operator touches a virtual control area 72a of touch pad 12 to 
highlight numerical entry "6" of numerical key pad 72 of on-screen control panel 

25 70. Similarly, virtual control areas 74a and 76a are mapped to corresponding to 
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cable chaimel control arrows 74 and volume control arrows 76 of on-screen 
control panel 70. As the operator touches different portions of touch pad 12 the 
corresponding portions of on-screen control panel 70 are highlighted. The 
operator then highlights the "ENTER" entry of on-screen control panel 70 or 
5 presses select button 24a of remote control 13 to select cable channel "6" . 

The operator may use numerical key pad 72 of on-screen control 
panel 70 to change the cable channel to view programming of another chaimel. 
The operator may use cable channel control arrows 74 to sequentially change the 
cable channels. The operator may use volume control arrows 76 to raise and 
10 lower the volume. 



At any time the operator may access main menu 30 by pressing 

main screen control button 24b of remote control 13. From main menu 30 the 

operator may select other HE device modes using operating panel 34. For 

example, the operator may select a player/recorder mode button 36 in operating 

15 panel 34 to permit control system 10 to access a player/recorder such as a tape 

device, VCR, optical disk drive, magnetic disk drive, or the like. For example, 

FIG. 7 illustrates a VCR program being displayed on video programming screen 

« 

52 of display screen 16 while the HE device is in the VCR mode. 

While viewing the video program on video programming screen 52 
20 the operator may access an on-screen VCR control panel 80 by selecting on-screen 

control button 24b of remote control 13. On-screen VCR control panel 80 
includes VCR control buttons 82 such as "PLAY", "STOP'', and the like. The 
area of touch pad 12 is mapped absolutely to the area of on-screen VCR control 
panel 80. Each VCR control button 82 of on-screen VCR control panel 80 
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corresponds to a respective portion of touch pad 12. The operator touches a 
portion of touch pad 12 to highlight the corresponding VCR control hutton 82 of 
on-screen VCR control panel 80. For example, the operator touches the middle 
portion of touch pad 12 to highlight the "REW" VCR control button 82 of on- 
5 screen VCR control panel 80. As the operator touches different portions of touch 

pad 12 the corresponding VCR control buttons 82 of on-screen VCR control panel 
80 are highlighted. In response to the operator selecting a VCR control button 82 
the HE device performs the VCR control function associated with the selected 
VCR control button 82. 

10 In addition to using on-screen controls 70 and 80 for enabling TV 

and VCR control functions, the operator may perform gestures on touch pad 12 
to enable the same TV and VCR control functions. A gesture performed on touch 
pad 12 is a stroke on the touch pad that corresponds to an understood or 
recognizable pattern. In response to a gesture performed on touch pad 12, the 

15 touch pad generates a gesture signal. The gesture signal is indicative of the 
gesture performed on touch pad 12. Each gesture performed on touch pad 12 
corresponds to a HE device control function. Controller 14 is operable with touch 
pad 12 for receiving a gesture signal from the touch pad in response to an operator 
performing'a gesture on the touch pad. In response to an operator performing a 

20 gesture on touch pad 12 controller 14 controls the HE device to enable the control 
function corresponding to the gesture performed on the touch pad. 

Gestures may be alphanumeric characters traced on touch pad 12. 
For instance, an operator may trace "P'' on touch pad 12 to change the television 
channel to channel "9". The operator may also trace "M** to mute the volume of 
25 the television or trace "P" to play the VCR. Gestures may also be a stroke across 
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touch pad 12. For example, a stroke from the right to the left of touch pad 12 
followed by a hold on the touch pad may correspond to a control function for 
turning up the volume of the television. A gesture may be a tap on touch pad 12. 
This gesture may correspond to stoppiug the VCR. 

Using gestures to control HE devices has many advantages. The 
operator has access to commands with no need to look at remote control 13. 
Gestures decrease the number of buttons needed on remote control 13. Remote 
control 13 can be upgraded simply by adding recognizable gestures. Hardware 
changes are not required meaning that there is no need to add, subtract, or change 
physical buttons or legends. 

In the Internet browser mode, the HE device ftinctions as an access 
device for Internet communications and display screen 16 disp]ays.an Internet page 
90 as shown in FIG. 8. In the Internet browser mode, a cursor 91 is displayed on 
Internet page 90 to enable the operator to navigate through the Internet page. 
15 Because the area of an Internet page is tj^ically bigger than the area of a display 
screen touch pad 12 is used in a relative mode when the HE device is in the 
Internet browser mode. In the relative mode, the operator moves cursor 91 by 
moving his finger across touch pad 12. Controller 14 moves cursor 91 to the 
location on Internet page 90 corresponding to the location of the touch on touch 
20 pad 12 in response to an operator touching the touch pad. 

Touch pad 12 may also function in the absolute mode when the HE 
device is in the Internet browser mode. In the absolute mode, the area of touch 
pad 12 is mapped absolutely to the area of Internet page 90. Each portion of 
Internet page 90 corresponds to a respective portion of touch pad 12. The 
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operator highlights links on the Internet page 90 by moving his finger to the 
corresponding portion of touch pad 12. The operator may then select a 
highlighted link by using select button 24a of remote control 13. 

While viewing Internet page 90 the operator may access an on- 
5 screen keyboard 92 by selecting on-screen control button 24b of remote control 

13. On-screen keyboard 92 includes the alphanumeric keys found on a typical 
computer keyboard. The area of touch pad 12 is napped absolutely to the area of 
on-screen keyboard 92. Each key of on-screen keyboard 92 corresponds to a 
respective portion of touch pad 12. The operator touches a portion of touch pad 

10 12 to highlight the corresponding key of on-screen keyboard 92. For example, the 

operator touches the upper right portion of touch pad 12 to highlight key "P" of 
on-screen keyboard 92. As the operator touches different portions of touch pad 
12 the corresponding keys of on-screen keyboard 92 are highlighted. On-screen 
keyboard 92 enables the operator to enter text into the input areas of Internet page 

15 90. To enter text, the operator highlights a key and then presses the select button 

24a of remote control 13 . Controller 14 then enters the selected text into the input 
area of Internet page 90. 

In addition to entering text by using on-screen keyboard 92, the 
operator may also enter text by writing letters using touch pad 12. As shown in 

20 FIG. 9, a graphical entry screen 94 is displayed on Internet page 90 in response 

to the operator pressing write button 24e of remote control 13. Graphical entry 
screen 94 displays letters entered by the operator touching touch pad 12. The area 
of touch pad 12 is mapped absolutely to the area of graphical entry screen 94. In 
operation, the operator touches touch pad 12 to sketch out a character such as the 

25 letter "R'' shown in FIG. 11. After sketching out the character the operator 
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presses the select button 24a of remote control 13 to enter the character into the 
input area of Internet page 90. Controller 14 uses character recognition software 
toxonvert the handwritten letter to text and then enters the text into the input area 
of Internet page 90. 

5 In the audio receiver mode, the HE device functions as a stereo 

receiver for playmg audio signals and display screen 16 displays an audio receiver 
control panel 100 as shown in FIG. 12A. Audio receiver control panel 100 
includes a receiver mode menu 102, a receiver control menu 104, and HE device 
operating panel 34. Receiver mode menu 102 includes a plurality of mode buttons 
10 106 corresponding to a receiver mode. For instance, modes of the receiver 

include AM and FM radio, tape, phonograph, compact disc, and the like. 
Receiver control menu 104 includes control buttons 108 for controlling the volume 
and changing the radio stations in the audio receiver mode. 

Referring now to FIG. 12B with continual reference to FIG. 12a, 
15 the area of audio receiver control panel 100 is mapped to the area of touch pad 
12. Each menu of audio receiver control panel 100 corresponds to a respective 
portion of touch pad 12. As shown in FIG. 12B, touch pad 12 is broken down 
into three control areas corresponding to receiver mode menu 102, receiver 
control menu 104, and HE device operating panel 34 of audio receiver control 
20 panel 100. Each control area of touch pad 12 has associated subareas 
corresponding to mode buttons 106, control buttons 108, and HE device selections 
of operating panel 34 in the audio receiver control panel 100. For instance, the 
operator touches a virtual control area 106a in touch pad 12 to highlight a mode 
button 106 in receiver mode menu 102, a virtual control area 108a to highlight a 
25 control button 108 in receiver control menu 104, and a virtual control area 34a to 
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highlight a selection of operating panel 34. The operator may then select a 
highlighted button using select button 24a of remote control 13. 

As mentioned above, main menu 30 further includes an email panel 
38 and an Internet telephony panel 40. An HE device such as an enhanced TV 
5 provides the operator with the ability to communicate email and Internet telephony 

messaging. In the Internet telephony mode, the HE device functions as a 
telephone. Display screen 16 displays an Internet telephony control panel 1 10 as 
shown in FIG. 13. Internet telephone control panel 110 includes a telephone 
directory menu 112, a telephone directory control panel 114, and a telephone 

10 operating panel 116. The area of telephone control panel 110 is mapped to the 

area of touch pad 12. Each menu of telephone control panel 110 corresponds to 
a respective portion of touch pad 12. For instance, the operator touches the upper 
left portion of touch pad 12 to highlight a telqphone entry 118 in telephone 
directory menu 1 12 . The operator may then select the highlighted telephone entry 

15 by using select button 24a of remote control 13. 

The operator may open received telephone messages by touching 
the area of touch pad 12 corresponding to telephone message area 120. In 
response, the HE device plays the recorded audible message. The operator may 
select a stored telephone number 118, dial the selected telephone number 122, talk 
20 and listen to the called party, and then hang up 124. To enter a telephone nmnber 

that is not stored, the operator selects dial 122 and then enters the desired 
telephone number using an on-screen numerical keyboard 130 displayed on display 
screen 16 as shown in FIG. 14. 
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In the email mode, the HE device commmiicates email messages. 
Display screen 16 displays an email panel 140 as shown in FIG. 15. Email panel 
140 includes an email directory menu 142. The area of email directory menu 142 
is mapped to the area of touch pad 12. Each email entry of email directory menu 
5 142 corresponds to a respective portion of touch pad 12. For instance, the 

operator touches the middle portion of touch pad 12 to highlight an email address 
entry 144 in email directory menu 142. The operator may then select the 
highlighted email entry by using select button 24a of remote control 13. 

Upon selecting the highlighted email entry display screen 16 
10 displays an email messaging panel 150 as shown in FIG. 16. The operator uses 

email messaging panel 150 to send email messages including text, drawings, voice 
clips, photographs, and the like. The area of touch pad 12 is mapped to email 
messaging panel 150 such that there is a one-to-one correspondence between each 
location of the touch pad and the email messaging panel in the same manner with 
15 reference to the other panels and menus described above. The operator may enter 

text by using on-screen keyboard 152 in the same manner described above with 
reference to the other on-screen keyboards. The operator may enter writing or 
drawings into a writing area 152 of email messaging panel 150 as shown in FIG. 
17 by using touch pad 12. Photographs may be copied to writing area 152 and 
20 then edited by using touch pad 12. In essence, touch pad 12 enables the operator 
to write an email message. For example, the operator may write his signature to 
electronically sign for purchases made via Internet shopping, on line banking, 
signature input, or simply handwrite a personalized email message. 

During operation of the HE device, the switching between the 
• 25 absolute mode and the relative mode is context sensitive. The operator will note 

-25^ 



DCX^ID: <WO__0178054A1_I_> 



wo 01/78054 



PCT/USOl/11682 



that during relative operation a cursor appears on display screen 16 and in absolute 
mode items, links, and the like will be highlighted when selected. The relative 
mode may be used for virtual screens that are larger than can be displayed on the 
display screen 16 at once. A typical application that uses the relative mode is the 
5 Internet browser. The functionality of a relative cursor allows for multiple strokes 

to move the cursor. However, if the cursor is against the sides of the display 
screen 16 the cursor will move the display like a window over the larger document 
(i.e. , pan and scroll) in the direction of the stroke across the touch pad 12. Once 
the cursor is placed over an object and the object is selected the mode may change 

10 based on the context. If the selected item were a button type the button is 

depressed. If the selected item is a text entry based on user preferences an on- 
screen keyboard of a character recognizer absolute menu pops up. Upon the input 
of text the system closes the text input screen and returns to the relative mode for 
the Internet browser. Once the Internet browser or other application requiring the 

15 larger display area is closed the menu would retum to the absolute mode. 

FIG. 18 illustrates a flow chart 160 describing operation of the 
switching between absolute and relative modes of display screen 16 according to 
an embodiment of the present invention. Flow chart 160 begins with receiving a 
new message as shown in block 162 and then incrementing the message count as 
20 shown in block 164. Decision block 166 then determines if touch pad 12 is 

activated. If not, the message count is set to zero and the relative move is set to 
zero as shown in block 168. The pad position is then equal to the last touch pad 
position as shown in block 170. 

If the touch pad is activated, then decision block 172 determines if 
25 the current mode is the absolute mode. If yes, then the display screen position is 
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equal to the touch pad position divided by the maximum touch pad position 
multiplied by the display size as shown in block 174. If not, then decision block 
176 determines if the message count is equal to one. If not, then the relative 
output is equal to the touch pad position minus the last touch pad position as 
5 ' shown in block 178. The touch pad position is then equal to the last touch pad 
position as shown in block 170. 

FIG. 19 illustrates in further detail remote control system 10 in 
accordance with the present invention. Remote control 13 includes control logic 
180 operable with touch pad 12, control buttons 24, and a transmitter 182. 

10 Control logic 180 is embodied as a microprocessor and memory. Controller 14 

includes control logic 184 operable with an audio/video processor 186 and a 
receiver 188. Receiver 188 receives signals from transmitter 182 and passes them 
to control logic 184. Control logic 184 is also embodied as a microprocessor and 
memory. In response to the signals received from transmitter 182, control logic 

15 184 controls audio/video processor 186 accordingly. In response, audio/video 
processor 186 controls display screen 16 and other controlled devices as a function 
of the received signals. 

Thus it is apparent that there has been provided, in accordance with 
the present invention, a remote control having a touch pad mapped to the screen 

20 of a home, entertainment (HE) device to create a virtual touch screen for enabling 
an operator to control the HE device that fully satisfies the objects, aims, and 
advantages set forth above. While the present invention has been described in 
conjunction with specific embodiments thereof, it is evident that many alternatives, 
modifications, and variations will be apparent to those skilled in the art in light of 

25 the foregoing description. Accordingly, it is intended to embrace all such 
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alternatives, modifications, and variations as fall within tlie spirit and broad scope 
of the appended claims. 
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What Is Claimed Is : 

1. A remote control for controlling a home entertainment (HE) 
device having a display screen, the remote control comprising: 

a touch pad mapped to the display screen of the HE device as a 
function of the ratio of the areas of the touch pad and the display screen such that 
each location of the touch pad corresponds to a respective location of the display 
screen; and 

a controller operable with the touch pad to enable a control function 
associated with a location of the display screen in response to the corresponding 
location of the touch pad being touched for controlling the HE device. 

2. The remote control of claim 1 wherein: 
the display screen includes control panels and each location of the 

touch pad corresponds to the control panel closest to the respective location of the 
display screen. 

15 3. The remote control of claim 2 wherein: 

at least one of the control panels includes control entries and the 
locations of the touch pad corresponding to the at least one of the control panels 
correspond to the control entries closest to the respective locations of the display 
screen. 

20 .4. The remote control of claim 1 wherein: 

the display screen includes a control panel having control functions, 
wherein the touch pad is mapped to the control panel of the display screen as a 
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function of the areas of the touch pad and the control panel such that each location 
of the touch pad corresponds to a respective control function of the control paneL 

5. The remote control of claim 1 wherein: 

the controller enables a control function in response to the 
5 corresponding location of the touch pad being touched. 

6. The remote control of claim 1 wherein: 

the display screen includes an electronic program guide having a 
plurality of grids, wherein the touch pad is mapped to the electronic program 
guide of the display screen as a function of the ratio of the areas of the touch pad 
10 and the electronic program guide such that each location of the touch pad 
corresponds to a respective grid of the electronic program guide. 

7. The remote control of claim 6 wherein: 

the controller enables a grid of the electronic program guide in 
response to the corresponding location of the touch pad being touched. 

15 8. The remote control of claim 1 wherein: 

the display screen includes an on-screen keyboard having 
alphanumeric keys, wherein the touch pad is mapped to the on-screen keyboard 
as a function of the ratio of the areas of the touch pad and the on-screen keyboard 
such fliat each location of the touch pad corresponds to a respective alphanumeric 

20 key of the on-screen keyboard. 

9. The remote control of claim 8 wherein: 
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the controller enables an alphaniuneric key of the on-screen 
keyboard in response to the corresponding location of the touch pad being 
touched. 

10. The remote control of claim 1 wherein: 

the display screen includes an Internet page having links, wherein 
the touch pad is mapped to the Internet page as a function of the ratio of the areas 
of the touch pad and the Internet page such that each location of die touch pad 
corresponds to a respective link of the Internet page. 

11. The remote control of claim 10 wherein: 

the controller enables a link of the Internet page in response to the 
corresponding location of the touch pad being touched. 

12. The remote control of claim 1 wherein: 

the display screen includes a graphical entry screen, wherein the 
touch pad is mapped to the graphical entry screen as a function of the ratio of the 
areas of the touch pad and the graphical entry screen such that each location of the 
touch pad corresponds to a respective location of the graphical entry screen. 

13- The remote control of claim 12 wherein: 
the controller draws on a location of the graphical entry screen in 
response to the corresponding location of the touch pad being touched. 



14. The remote control of claim 1 wherein: 
the display screen includes an email panel, wherein the touch pad 
is mapped to the email panel as a function of the ratio of the areas of the touch pad 
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and the email panel such that each location of the touch pad corresponds to a 
respective location of the email panel. 

15. The remote control of claim 1 wherein: 

the display screen includes an telephony panel, wherein the touch 
5 pad is mapped to the telephony panel as a function of the ratio of the areas of the 
touch pad and the telephony panel such that each location of the touch pad 
corresponds to a respective location of the telephony panel. 

16. A remote control for controlling a home entertainment device 
having an electronic program guide displayed on a display screen, the electronic 

10 program guide having a plurality of grids, the remote control comprising: 

a touch pad mapped to the electronic program guide as a function 
of the ratio of tiie areas of the touch pad and the electronic program guide such 
that each location of the touch pad corresponds to a respective grid of the 
electronic program guide. 

15 17. The remote control of claim 16 wherein: 

the touch pad is operable with a controller for enabling a grid of the 
electronic program guide in response to the corresponding location of the touch 
pad being touched. 

18 . A remote control for controlling a home entertainment device 
20 having an on-screen keyboard displayed on a display screen, the on-screen 
keyboard having a plurality of alphanumeric keys, the remote control comprising: 
a touch pad mapped to the on-screen keyboard as a function of the 
ratio of the areas of the touch pad and the on-screen keyboard such that each 

-32- 



SDOCID: <WO_0178054A1J_> 



wo 01/78054 



PCT/USOl/11682 



location of the touch pad corresponds to a respective alphanumeric key of the on- 
screen keyboard. 

19. The remote control of claim 18 wherein: 

the touch pad is operable with a controller for enabling an 
5 alphanumeric key of the on-screen keyboard in response the corresponding 
location of the touch pad being touched. 

20 . A remote control for controlling a home entertainment device 
having an Internet page displayed on a display screen, the Internet page having a 
plurality of links, the remote control comprising: 

10 a touch pad mapped to the Internet page as a function of the ratio 

of the areas of the touch pad and the Internet page such that each location of the 
touch pad corresponds to a respective link of the Internet page. 

21. The remote control of claim 20 wherein: 

the touch pad is operable with a controller for enabling a link of the 
15 Internet page in response to the corresponding location of the touch pad being 
touched. 

22. A remote control for controlling a television, the remote 
control comprisiQg: 

a touch pad; and 

20 a plurality of control buttons, wherein the touch pad and the control 

buttons are used in conjunction to enable control functions for controlling the 
television. 
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23. The remote control of claim 22 wherein: 

the touch pad is mapped to the display screen of the television as 
a function of the ratio of the areas of the touch pad and display screen such that 
each location of the touch pad corresponds to a respective location of the display 
screen, wherein a control function associated with a location of the display screen 
is selected by touching the corresponding location of the touch pad and then 
enabled using a control button. 

24. A mefliod for controlling a home entertainment (HE) device 
having a display screen, the method comprising: 

touching a portion of a touch pad, wherein the touch pad includes 
a plurality of virtual control areas each associated with a corresponding virtual 
control on the display screen, wherein each portion of the touch pad is associated 
with a virtual control area of the touch pad; and 

enabling a virtual control displayed on the display screen 
corresponding to the virtual control area of the touch pad associated with the 
touched portion of the touch pad. 

25. The method of claim 24 further comprising: 

controlling the HE device as a function of the enabled virtual 

control. 

26. The method of claim 24 wherein: 

each virtual control area of the touch pad is associated with a 
corresponding virtual control on the display screen as a function of the positions 
of the virtual control areas on the touch pad and the positions of the virtual 
controls on the display screen. 
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27. The method of claim 24 wherein: 

the display screen includes an electronic program guide having a 
plurality of virtual control grid areas, wherein the touch pad is mapped to the 
electronic program guide as a function of the ratio of the areas of the touch pad 
5 and the electronic program guide such that each virtual control area of the touch 
pad is associated with a corresponding virtual control grid area of the electronic 
program guide. 

28. The method of claim 27 wherein: 

enabling a virtual control includes enabling a virtual control grid 
10 area of the electronic program guide corresponding to the virtual control area of 
the touch pad associated with the touched portion of the touch pad. 

29. The method of claim 24 wherein: 

the display screen includes an on-screen keyboard having a plurality 
of virtual control alphanumeric keys, wherein the touch pad is mapped to the on- 
15 screen keyboard as a function of the ratio of the areas of the touch pad and the on- 
screen keyboard such that each virtual control area of the touch pad is associated 
with a corresponding virtual control alphanmneric key of the on-screen keyboard. 

30. The method of claim 29 wherein: 

enabling a virtual control includes enabling a virtual control 
20 alphanumeric key of the on-screen keyboard corresponding to the virtual control 
area of the touch pad associated with the touched portion of the touch pad. 

3 1 . The method of claim 24 wherein: 
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the display screen includes an Internet page having a plurality of 
Virtual control links, wherein the touch pad is mapped to the Internet page as a 
function of the ratio of the areas of the touch pad and the Internet page such that 
each virtual control area of the touch pad is associated with a corresponding 
5 virtual control link of the Internet page. 



32, The method of claim 31 wherein: 

enabling a virtual control includes enabling a virtual control link of 
the Internet page corresponding to the virtual control area of the touch pad 
associated with the touched portion of the touch pad. 

10 33. The method of claim 24 wherein: 

the display screen includes an email panel having a plurality of 
virtual controls, wherein the touch pad is mapped to the email panel as a function 
of the ratio of the areas of the touch pad and the email panel such that each virtual 
control area of the touch pad is associated with a corresponding virtual control of 

15 the email panel. 

34. The method of claim 33 wherein: 

enabling a virtual control includes enabling a virtual control of the 
email panel corresponding to the virtual control area of the touch pad associated 
with the touched portion of the touch pad. 

20 35. The method of claim 24 wherein: 

the display screen includes a telephony panel having a plurality of 
virtual controls, wherein the touch pad is mapped to the telephony panel as a 
function of the ratio of the areas of the touch pad and the telephony panel such that 
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each virtual control area of the touch pad is associated with a corresponding 
virtual control of the telephony panel. 

36. A method for controlling a home entertainment (HE) device 
having an electronic program guide displayed on a display screen, the electronic 

5 program guide having a plurality of grid areas, the method comprising: 

touching a portion of a touch pad, wherein the touch pad includes 
a plurality of virtual control areas each associated with a corresponding grid area 
of the electronic program guide, wherein each portion of the touch pad is 
associated with a virtual control area of the touch pad; and 
10 enabling a grid area of the electronic program guide displayed on 

the display screen corresponding to the virtual control area of the touch pad 
associated with the touched portion of the touch pad. 

37. The method of claim 36 further comprising: 

controlling flie HE device as a function of the enabled grid area of 
15 the electronic program guide. 

38. A method for controlling a home entertainment (HE) device 
having an on-screen keyboard displayed on a display screen, the on-screen 
keyboard having a plurality of alphanumeric keys, the method comprising: 

touching a portion of a touch pad, wherein the touch pad includes 
20 a plurality of virtual control areas each associated with a corresponding 
alphanumeric key of the on-screen keyboard, wherein each portion of the touch 
pad is associated with a virtual control area of the touch pad; and 
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enabling an alphaniimeric key of the on-screen keyboard 
corresponding to the virtual control area of the touch pad associated with the 
touched portion of the touch pad. 

39. The method of claim 38 further comprising: 

5 controlling the HE device as a function of the enabled alphanumeric 

key of the on-screen keyboard. 

40. A method for controlling a home entertainment (HE) device 
having an Internet page displayed on a display screen, the Internet page having a 
plurality of links, the method comprising: 

10 touching a portion of a touch pad, wherein the touch pad includes 

a plurality of virtual control areas each associated with a corre^>onding link of the 
Internet page, wherein each portion of the touch pad is associated with a virtual 
control area of the touch pad; and 

enabling a link of the Memet page corresponding to the virtual 

15 control area of the touch pad associated with the touched portion of the touch pad. 

41 . The method of claim 40 further comprising: 
controlling the HE device as a function of the enabled link of the 

Internet page. 
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